Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(1255): Integrated the AWS costexplorer API #1278

Merged
merged 7 commits into from
Dec 18, 2023

Conversation

tanbirali
Copy link
Contributor

Problem

There are few AWS Services that are pending integration

Solution

Filter out the correct resource and add to metdata field

Changes Made

User the GetCostAndUsage to extract the cost and then added it to the metadata for the following services API for ec2, ecs, iam, s3, sqs, cloudfront, eks, elasticahce, redshift

How to Test

[Provide instructions on how to test the changes you made, including any relevant details like configuration steps or data to be used for testing.]

Reviewers

@Azanul

… sqs, cloudfront, eks, elasticahce, redshift
@Azanul
Copy link
Collaborator

Azanul commented Dec 12, 2023

Good work @tanbirali! It'd be great if you could implement the other ones also, if not then this PR looks GTG

@tanbirali
Copy link
Contributor Author

I will be happy to contribute more

@jakepage91
Copy link
Contributor

Thanks @tanbirali ! Much appreciated

providers/aws/sns/topics.go Outdated Show resolved Hide resolved
@Azanul Azanul requested review from jakepage91 and Azanul December 13, 2023 06:39
Copy link
Collaborator

@Azanul Azanul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although I haven't cross checked the service identifiers used (whether they're supported and whether they're supported under this name), I think we're GTG.

Copy link
Contributor

@jakepage91 jakepage91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @tanbirali, have a few questions though.

providers/aws/dynamodb/tables.go Outdated Show resolved Hide resolved
providers/aws/systemsmanager/maintenance_window.go Outdated Show resolved Hide resolved
providers/aws/apigateway/apis.go Outdated Show resolved Hide resolved
providers/aws/cloudfront/distributions.go Outdated Show resolved Hide resolved
@Azanul Azanul requested a review from jakepage91 December 13, 2023 12:56
Copy link
Contributor

@jakepage91 jakepage91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there @tanbirali , it looks like a lot of the service names do not follow the same naming returned by the costexplorer api, here is an snippet of the return

{
    "DimensionValues": [
        {
            "Value": "AWS CloudTrail",
            "Attributes": {}
        },
        {
            "Value": "AWS Config",
            "Attributes": {}
        },
        {
            "Value": "AWS Cost Explorer",
            "Attributes": {}
        },
        {
            "Value": "AWS Data Transfer",
            "Attributes": {}
        },
        {
            "Value": "AWS Glue",
            "Attributes": {}
        },
        {
            "Value": "AWS Key Management Service",
            "Attributes": {}
        },
        {
            "Value": "AWS Lambda",
            "Attributes": {}
        },
        {
            "Value": "AWS Migration Hub Refactor Spaces",
            "Attributes": {}
        },
        {
            "Value": "AWS Secrets Manager",
            "Attributes": {}
        },
        {
            "Value": "AWS Service Catalog",
            "Attributes": {}
        },
        {
            "Value": "AWS Step Functions",
            "Attributes": {}
        },

Now, it might be the case that you have tested this already and the current service names work and actually return the service level cost metrics, could you confirm that to be the case? or else could you please change the names to the aws ce get-dimension-values call.

@tanbirali
Copy link
Contributor Author

I have made the changes of every aws services that I could find in that text file that contained the proper service name. But there are some services whose name were not present like for s3, codebuild, codedeploy etc. Let me if I need to make changes there.

@Azanul
Copy link
Collaborator

Azanul commented Dec 18, 2023

I have made the changes of every aws services that I could find in that text file that contained the proper service name. But there are some services whose name were not present like for s3, codebuild, codedeploy etc. Let me if I need to make changes there.

S3 = "Amazon Simple Storage Service"
codebuild = "AWS CodeBuild"
codedeploy = "AWS CodeCommit"

I think these will work and if not we can use SERVICE_CODEs instead of SERVICE dimension. I've listed all the service codesthat I could find here

Copy link
Contributor

@jakepage91 jakepage91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Azanul Azanul merged commit bc25338 into tailwarden:develop Dec 18, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants